Task-Directed Exploration in Continuous POMDPs for Robotic Manipulation of Articulated Objects

Representing and reasoning about uncertainty is crucial for autonomous agents acting in partially observable environments with noisy sensors. Partially observable Markov decision processes (POMDPs) serve as a general framework for representing problems in which uncertainty is an important factor. Online sample-based POMDP methods have emerged as efficient approaches to solving large POMDPs and have been shown to extend to continuous domains. However, these solutions struggle to find long-horizon plans in problems with significant uncertainty. Exploration heuristics can help guide planning, but many real-world settings contain significant task-irrelevant uncertainty that might distract from the task objective. In this paper, we propose STRUG, an online POMDP solver capable of handling domains that require long-horizon planning with significant task-relevant and task-irrelevant uncertainty. We demonstrate our solution on several temporally extended versions of toy POMDP problems as well as robotic manipulation of articulated objects using a neural perception frontend to construct a distribution of possible models. Our results show that STRUG outperforms the current sample-based online POMDP solvers on several tasks

EARL: Eye-on-Hand Reinforcement Learner for Dynamic Grasping with Active Pose Estimation

In this paper, we explore the dynamic grasping of moving objects through active pose tracking and reinforcement learning for hand-eye coordination systems. Most existing vision-based robotic grasping methods implicitly assume target objects are stationary or moving predictably. Performing grasping of unpredictably moving objects presents a unique set of challenges. For example, a pre-computed robust grasp can become unreachable or unstable as the target object moves, and motion planning must also be adaptive. In this work, we present a new approach, Eye-on-hAnd Reinforcement Learner (EARL), for enabling coupled Eye-on-Hand (EoH) robotic manipulation systems to perform real-time active pose tracking and dynamic grasping of novel objects without explicit motion prediction. EARL readily addresses many thorny issues in automated hand-eye coordination, including fast-tracking of 6D object pose from vision, learning control policy for a robotic arm to track a moving object while keeping the object in the camera's field of view, and performing dynamic grasping. We demonstrate the effectiveness of our approach in extensive experiments validated on multiple commercial robotic arms in both simulations and complex real-world tasks.Comment: Presented on IROS 2023 Corresponding author Siddarth Jai

Style-transfer based Speech and Audio-visual Scene Understanding for Robot Action Sequence Acquisition from Videos

To realize human-robot collaboration, robots need to execute actions for new tasks according to human instructions given finite prior knowledge. Human experts can share their knowledge of how to perform a task with a robot through multi-modal instructions in their demonstrations, showing a sequence of short-horizon steps to achieve a long-horizon goal. This paper introduces a method for robot action sequence generation from instruction videos using (1) an audio-visual Transformer that converts audio-visual features and instruction speech to a sequence of robot actions called dynamic movement primitives (DMPs) and (2) style-transfer-based training that employs multi-task learning with video captioning and weakly-supervised learning with a semantic classifier to exploit unpaired video-action data. We built a system that accomplishes various cooking actions, where an arm robot executes a DMP sequence acquired from a cooking video using the audio-visual Transformer. Experiments with Epic-Kitchen-100, YouCookII, QuerYD, and in-house instruction video datasets show that the proposed method improves the quality of DMP sequences by 2.3 times the METEOR score obtained with a baseline video-to-action Transformer. The model achieved 32% of the task success rate with the task knowledge of the object.Comment: Accepted to Interspeech202

EGFR Dynamics Change during Activation in Native Membranes as Revealed by NMR

The epidermal growth factor receptor (EGFR) represents one of the most common target proteins in anti-cancer therapy. To directly examine the structural and dynamical properties of EGFR activation by the epidermal growth factor (EGF) in native membranes, we have developed a solid-state nuclear magnetic resonance (ssNMR)-based approach supported by dynamic nuclear polarization (DNP). In contrast to previous crystallographic results, our experiments show that the ligand-free state of the extracellular domain (ECD) is highly dynamic, while the intracellular kinase domain (KD) is rigid. Ligand binding restricts the overall and local motion of EGFR domains, including the ECD and the C-terminal region. We propose that the reduction in conformational entropy of the ECD by ligand binding favors the cooperative binding required for receptor dimerization, causing allosteric activation of the intracellular tyrosine kinase

EGFR Dynamics Change during Activation in Native Membranes as Revealed by NMR

The epidermal growth factor receptor (EGFR) represents one of the most common target proteins in anti-cancer therapy. To directly examine the structural and dynamical properties of EGFR activation by the epidermal growth factor (EGF) in native membranes, we have developed a solid-state nuclear magnetic resonance (ssNMR)-based approach supported by dynamic nuclear polarization (DNP). In contrast to previous crystallographic results, our experiments show that the ligand-free state of the extracellular domain (ECD) is highly dynamic, while the intracellular kinase domain (KD) is rigid. Ligand binding restricts the overall and local motion of EGFR domains, including the ECD and the C-terminal region. We propose that the reduction in conformational entropy of the ECD by ligand binding favors the cooperative binding required for receptor dimerization, causing allosteric activation of the intracellular tyrosine kinase

Nations within a nation: variations in epidemiological transition across the states of India, 1990–2016 in the Global Burden of Disease Study

18% of the world's population lives in India, and many states of India have populations similar to those of large countries. Action to effectively improve population health in India requires availability of reliable and comprehensive state-level estimates of disease burden and risk factors over time. Such comprehensive estimates have not been available so far for all major diseases and risk factors. Thus, we aimed to estimate the disease burden and risk factors in every state of India as part of the Global Burden of Disease (GBD) Study 2016